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Abstract.  How  do  experienced  users  extract  information  from  a  com¬ 
plex  visualization?  We  examine  this  question  by  presenting  experienced 
weather  forecasters  with  visualizations  that  did  not  show  the  needed  in¬ 
formation  explicitly  and  examining  their  eye  movements.  We  replicated 
Carpenter  &  Shah  (1998)  when  the  information  was  explicitly  available 
on  the  visualization.  However,  when  the  information  was  not  explicitly 
available,  we  found  that  forecasters  used  spatial  reasoning  in  the  form 
of  spatial  transformations.  We  also  found  a  strong  imagerial  component 
for  constructing  meteorological  information. 


1  Introduction 

How  do  people  comprehend  and  extract  information  from  a  complex  graph  or  vi¬ 
sualization?  Current  models  of  graph  comprehension  account  very  well  for  simple 
tasks  (e.g.,  “What  is  the  price  of  graphium  in  1982?”).  Similarly,  most  empirical 
studies  have  used  simple  graphs  (e.g.,  few  variables  and  few  data  points).  For  ex¬ 
ample,  within  the  graph  comprehension  literature,  Lohse  (1993)  studied  some  of 
the  most  complex  graphs,  and  his  standard  graphs  used  6  variables  with  a  total 
of  72  data  points;  Carpenter  &  Shah  (1998)  provided  participants  with  graphs 
that  had  2  to  4  variables  with  4  to  8  data  points  (containing  interactions);  Tan  & 
Benbasat  (1990)  had  participants  study  graphs  that  displayed  3  variables  with 
18  total  data  points. 

In  contrast,  many  domains  attempt  to  display  tens  of  variables  and  tens 
of  thousands  of  data  points  (or  more).  For  example,  many  scientists  use  ex¬ 
tremely  complex  visualizations  with  thousands  of  data  points  (Trafton,  Trickett, 
&  Mintz,  in  press;  Trickett,  Fu,  Schunn,  &  Trafton,  2000;  Trickett,  Trafton, 
Schunn,  &  Harrison,  2001)  and  weather  forecasters  routinely  examine  visual¬ 
izations  with  10  or  more  variables  and  thousands  of  data  points  over  multiple 
spatial  and  temporal  scales  (Trafton,  Kirschenbaum,  Tsui,  Miyamoto,  Balias,  & 
Raymond,  2000).  Current  wisdom  suggests  that  showing  a  graph  well  depends 
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on  making  the  variables  and  patterns  explicit  (Gillan,  Wickens,  Hollands,  & 
Carswell,  1998;  Tufte,  1983,  1990,  1997).  However,  when  there  are  an  extremely 
large  number  of  data  points,  this  explicitness  rule  may  need  to  be  relaxed  in 
order  to  avoid  visual  clutter. 

We  believe  that  as  a  direct  result  of  the  complexity  of  the  graphs,  some  data 
will  need  to  be  represented  more  imprecisely  (e.g.,  pressure  on  Figure  1)  and 
this  imprecision  may  prevent  a  user  from  directly  reading  off  information  from 
the  graph.  The  user  may  need  to  use  a  different  set  of  mental  procedures  than 
is  needed  in  simpler  graphs. 

This  paper  first  reviews  current  models  of  graph  comprehension,  suggests 
augmenting  them  in  a  specific  manner  when  the  visualization  is  complex  and 
the  task  requires  information  that  is  implicitly  represented,  and  presents  an 
initial  study  to  explore  these  issues. 


1.1  Current  models  of  graph  comprehension 

The  most  influential  research  on  graph  and  visualization  comprehension  is  Bertin’s 
(1983)  task  analysis  which  suggests  three  main  processes  in  graph  and  visualiza¬ 
tion  comprehension: 

1.  Encode  visual  elements  of  the  display:  For  example,  identifying  lines  and 
axes.  This  stage  is  influenced  by  pre-attentive  processes  and  is  affected  by  the 
discriminability  of  shapes. 

2.  Translate  the  elements  into  patterns:  For  example,  notice  that  one  bar  is 
taller  than  another  or  the  slope  of  a  line.  This  stage  is  affected  by  distortions  of 
perception  and  limitations  of  working  memory. 

3.  Map  the  patterns  to  the  labels  to  interpret  the  specific  relationships  com¬ 
municated  by  the  graph.  For  example,  determine  the  value  of  a  bar  graph. 

Most  of  the  work  done  on  graph  comprehension  has  examined  the  encoding, 
perception,  and  representation  of  graphs.  Cleveland  and  McGill,  for  example, 
have  examined  the  psychophysical  aspects  of  graphical  perception  (Cleveland  & 
McGill,  1984,  1986).  Similarly,  Pinker’s  theory  of  graph  comprehension,  while 
quite  broad,  focuses  on  the  encoding  and  understanding  of  graphs  (Pinker,  1990). 
Kosslyn’s  work  emphasizes  the  cognitive  processes  that  make  a  graph  more  or 
less  difficult  to  read.  Kosslyn’s  syntactic  and  semantic  (and  to  a  lesser  degree 
pragmatic)  level  of  analysis  focuses  on  encoding,  perception,  and  representation 
of  graphs  (Kosslyn,  1989).  Tracking  users’  eye  movements,  Carpenter  &  Shah 
(1998)  have  shown  that  people  switch  between  looking  at  the  graph  and  the  axes 
in  order  to  comprehend  the  visualization.  Similarly,  Peebles  &  Cheng  (2001a, 
2001b)  have  suggested  that  in  experimental  trials,  people  cycle  between  looking 
at  the  graph  and  the  question  they  are  trying  to  answer 

This  scheme  seems  to  work  very  well  when  the  graph  explicitly  represents 
the  needed  information.  Thus,  when  a  meteorologist  is  asked  to  provide  the  wind 
direction  at  Pittsburgh  on  Figure  1,  the  meteorologist  searches  for  Pittsburgh, 
finds  the  wind  barb  over  Pittsburgh,  and  determines  the  direction  it  points  (280, 
or  westerly,  in  this  case).  Note  that  it  is  slightly  difficult  to  see  the  part  of  the 


barb  showing  wind  speed  in  this  figure,  which  should  not  affect  the  perception 
of  wind  direction. 
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Fig.  1.  A  typical  visualization  that  meteorologists  use.  Wind  speed  and  wind  direction 
is  shown  by  the  wind  barbs;  temperature  is  color  coded  to  the  legend  on  the  far  right, 
and  pressure  is  shown  by  lines  connecting  the  same  pressure  (1008,  1012,  etc.).  The 
original  is  in  color. 


1.2  When  information  is  not  explicitly  available 

What  happens  when  information  is  not  explicitly  available  on  a  graph?  For 
example,  in  Figure  1,  how  would  the  pressure  at  Pittsburgh  be  determined?  1 
Current  theories  either  do  not  deal  with  how  information  is  extracted  when 
the  information  is  not  explicitly  available  (e.g.,  Lohse,  1993)  or  leave  the  whole 
process  unspecified  (e.g.,  Pinker,  1990;  Kosslyn,  1989). 

Pinker’s  (1990)  framework,  for  example,  provides  the  most  detailed  descrip¬ 
tion  of  how  information  is  extracted  from  a  graph.  He  claims  that  once  the  graph 

1  If  a  pressure  line  goes  directly  through  the  location  for  which  a  forecaster  wants 
pressure,  the  forecaster  simply  reads  it  off  (e.g.,  St.  Louis’  pressure  is  1012).  If, 
however,  a  pressure  line  does  not  pass  directly  through  the  desired  location,  the 
forecaster  must  interpolate  between  two  lines  (e.g.,  Pittsburgh’s  pressure  is  1011 
because  it  is  between  1008  and  1012  and  closer  to  1012  than  the  middle). 


has  been  encoded,  all  inferences  and  reasoning  occur  propositionally.  Pinker’s 
“conceptual  questions”  and  “conceptual  messages”  are  his  primary  method  of 
inferencing  and  reasoning.  Thus,  Pinker’s  theory  suggests  that  people  initially 
encode  the  aspects  of  a  graph  into  a  propositional  representation,  and  then  rea¬ 
son  with  that  propositional  representation  to  answer  conceptual  questions.2 

Trafton  &  Trickett  (2001),  in  contrast,  suggested  that  a  great  deal  of  com¬ 
plex  visualization  reasoning  occurs  spatially,  especially  when  the  information  is 
not  explicitly  available  on  the  visualization.  Trafton  &  Trickett  (2001)  showed 
that  when  scientists’  own  complex  visualizations  did  not  explicitly  show  the  in¬ 
formation  they  needed,  scientists  created  complex  mental  representations  and 
manipulated  them  to  help  them  answer  the  specific  questions  they  had.  Trafton 
&  Trickett  suggested  that  the  scientists  constructed  spatial  representations. 

Trafton  &  Trickett  (2001)  suggested  that  when  people  who  used  complex 
visualizations  needed  to  extract  information  that  was  not  available,  they  used 
spatial  information  to  create  internal  mental  representations  to  reason  with.  We 
have  developed  a  framework  for  coding  and  working  with  these  kinds  of  graphs 
and  visualizations  called  Spatial  Transformations  that  will  be  used  to  investigate 
these  issues.  We  will  argue  that  spatial  transformations  are  a  fundamental  aspect 
of  complex  visualization  usage. 

Spatial  Transformations  are  cognitive  operations  that  a  scientist  performs 
on  a  visualization.  Sample  spatial  transformations  are  mental  rotation  (e.g., 
Shepard  &  Metzler,  1971),  creating  a  mental  image,  modifying  that  mental  image 
by  adding  or  deleting  features  to  or  from  it,  animating  an  aspect  of  a  visualization 
(Hegarty,  1992)  time  series  progression  prediction,  mentally  moving  an  object, 
mentally  transforming  a  2D  view  into  a  3D  view  (or  vice  versa),  comparisons 
between  different  views  (Kosslyn,  Sukel,  &  Bly,  1999;  Trafton  et  al.,  in  press),  and 
anything  else  a  scientist  mentally  does  to  a  visualization  in  order  to  understand 
it  or  facilitate  problem  solving.  Also  note  that  a  spatial  transformation  can  be 
done  on  either  an  internal  (i.e.,  mental)  representation  or  an  external  image 
(i.e.,  a  scientific  visualization  on  a  computer-generated  image).  A  more  complete 
description  of  spatial  transformations  (along  with  a  mini-experiment  to  teach 
interface  designers  when  to  use  a  2D  or  a  3D  representation)  can  be  found  at 
http : / / e-lab . hf ac . gmu . edu/ "trafton/ 405st . html 

Trafton  &  Trickett  (2001)  focused  on  how  scientists  created  mental  repre¬ 
sentations  to  answer  specific  questions.  Because  that  study  examined  scientists 
working  in  vivo  (Baker  &  Dunbar,  2000;  Dunbar,  1995,  1997;  Trickett  et  al., 
2001),  it  was  not  possible  to  control  what  they  saw  or  what  information  was 
needed.  In  this  study,  we  asked  experienced  meteorologists  to  provide  specific 
information  from  weather  visualizations  like  that  shown  in  Figure  1.  We  also 
tracked  where  they  were  looking  with  an  eyetracker  system. 

2  There  are,  of  course,  a  large  number  of  theories  that  discuss  diagrammatic  reasoning 
and  suggest  that  reasoning  occurs  spatially  or  via  images  (Hegarty  &  Sims,  1994; 
Larkin  &  Simon,  1987;  Narayanan  &  Hegarty,  1998;  Tabachneck-Scliijf,  Leonardo, 
&  Simon,  1997).  However,  most  of  these  theories  are  concerned  with  diagram  under¬ 
standing,  not  graph  comprehension. 


We  had  several  goals  in  this  study.  First,  we  wanted  to  provide  a  baseline 
measure  by  showing  that  when  experienced  users  extracted  information  explicitly 
available  from  a  complex  visualization,  they  would  extract  information  based  on 
the  canonical  graph  comprehension  model  presented  earlier;  this  would  also  be 
a  partial  replication  of  other  graph  comprehension  studies  (Carpenter  &  Shah, 
1998).  For  a  standard  meteorological  visualization,  we  predicted  that  experi¬ 
enced  meteorologists  would  be  able  to  go  directly  to  the  desired  information  and 
extract  it.  Second,  we  wanted  to  examine  how  forecasters  extract  information 
from  the  graph  that  was  not  explicitly  available  (i.e.,  do  they  use  a  propositional 
representation  or  do  they  reason  with  the  graph  itself).  We  complete  our  dis¬ 
cussion  with  some  anecdotal  evidence  of  how  forecasters  remember  information 
that  they  had  seen  previously. 

2  Method 

In  order  to  investigate  the  issues  discussed  above,  we  examined  forecasters  as 
they  were  examining  meteorological  visualizations. 

2.1  Task 

Forecasters  were  presented  with  a  weather  visualization  (see  Figure  1).  They 
were  then  asked  several  questions  in  the  following  order  during  the  Graph  Com¬ 
prehension  portion  of  this  study. 

—  What  is  the  synoptic  weather? 

—  What  is  the  wind  speed  and  wind  direction  at  Pittsburgh,  Pennsylvania? 

—  What  is  the  temperature  at  Pittsburgh,  Pennsylvania? 

—  What  is  the  pressure  at  Pittsburgh,  Pennsylvania? 

Forecasters  were  then  shown  a  second  weather  visualization  (see  Figure  2) 
and  asked  two  questions: 

—  What  is  the  wind  speed  and  wind  direction  at  Honolulu? 

—  What  is  the  relative  humidity  at  Honolulu? 

Finally,  during  the  recall  portion  of  the  study,  forecasters  were  shown  a  blank 
screen  and  asked  about  wind  speed,  wind  direction,  temperature,  and  pressure 
at  four  locations  from  the  first  visualization:  Pittsburgh,  Pennsylvania  (Exact), 
Philadelphia,  Pennsylvania  (Near),  Atlanta,  Georgia  (Medium),  and  Houston, 
Texas  (Far).  The  exact  question  probed  participants’  memory  for  the  information 
they  had  recently  read  off.  The  near,  medium,  and  far  conditions  allowed  us  to 
see  how  their  accuracy  changed  as  the  location  moved  farther  away  from  the  area 
they  focused  on  (Pittsburgh);  Philadelphia  is  very  close  to  Pittsburgh,  Georgia 
is  further  away,  and  Houston  is  quite  far  from  Georgia  on  the  visualization. 

It  should  be  noted  that  all  of  these  question  types  were  based  on  the  types 
of  tasks  meteorologists  typically  do  (Lowe,  1999;  Trafton  et  al.,  2000). 

These  questions  came  in  several  forms: 
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Fig.  2.  A  typical  visualization  that  meteorologists  use.  The  wind  barbs  represent  wind 
speed  and  direction;  the  black  lines  represent  geopotential  height,  and  the  dial  in  the 
upper  right  hand  corner  shows  a  legend  for  relative  humidity.  The  original  is  in  color. 


—  Questions  asking  for  qualitative  information,  where  the  users  have  to  inte¬ 
grate  across  information.  The  synoptic  weather  question  is  an  example  of 
this. 

—  Questions  asking  for  quantitative  information,  where  the  answers  (numbers) 
are  explicitly  represented  (as  in  traditional  graph  comprehension  studies). 
The  question  about  temperature  and  the  relative  humidity  question  fall  into 
this  category  because  users  only  have  to  map  a  color  to  the  legend  and  the 
numbers  are  printed  on  the  legend.  Also,  all  locations  on  the  map  have  an 
instance  of  the  variable.  We  expect  participants  to  read-off  the  information 
as  they  need  it,  just  as  the  graph  comprehension  literature  predicts. 

—  Questions  asking  for  quantitative  information  that  is  imprecisely  represented 
in  the  graph.  In  other  words,  the  numbers  themselves  are  not  represented 
explicitly,  but  there  is  a  symbology  associated  with  them  that  the  user  must 
know  in  order  to  extract  the  needed  information.  Wind  direction  and  wind 
speed  are  part  of  this  question  type.  If  the  user  has  the  knowledge  about  how 
to  interpret  the  symbology,  the  graph  comprehension  theories  would  predict 
a  read-off  strategy. 

—  Questions  asking  for  quantitative  information  that  is  entirely  implicit  in 
the  graph.  Numbers  are  not  represented  either  explicitly  or  imprecisely  to 
answer  that  question,  but  must  be  inferred.  The  question  about  pressure 
is  an  instance  of  this  category.  Although  pressure  is  represented  explicitly 


for  some  locations,  it  must  be  inferred  for  others  (like  Pittsburgh).  Graph 
comprehension  theories  do  not  make  good  predictions  here,  but  the  spatial 
transformation  theory  does  suggest  a  framework  for  which  this  information 
could  be  extracted. 

—  Questions  asking  for  quantitative  information  when  the  information  is  no 
longer  available  externally,  but  must  be  retrieved  from  memory.  All  the  recall 
questions  fall  into  this  category. 


2.2  Participants 

All  forecasters  were  Naval  or  Marine  Corps  forecasters  and  forecasters-in-training. 
All  had  completed  at  least  the  first  level  weather  school.  They  ranged  in  fore¬ 
casting  experience  from  1  to  10  years.  All  forecasters  had  significant  operational 
experience. 

Four  novices  and  two  expert  forecasters  performed  the  tasks.  The  experts  had 
an  average  of  10  years  forecasting  experience  and  the  novices  had  an  average  of 
2.5  years  forecasting  experience. 

2.3  Setting  and  Apparatus 

Meteorological  visualizations  were  collected  from  http :  //www.  fnmoc  .navy  .mil/. 
This  web  site  is  used  a  great  deal  by  current  Navy  forecasters;  all  forecasters  in 
the  study  were  very  familiar  with  this  site  and  the  visualizations  that  were  used. 
This  web  site  was  used  extensively  by  forecasters  in  a  previous  study  of  Navy 
meteorologists  (Trafton  et  al.,  2000) 

The  experimental  sessions  took  place  in  a  room  equipped  with  a  PC  and  an 
EyeLink  System  from  SMI.  The  eyetracker  had  headmounted  optics  with  three 
small  cameras  (left  eye,  right  eye,  head  compensation).  It  records  eye  movements 
and  pupil  dilations  at  250  Hz  (every  4  milliseconds). 

2.4  Procedure 

Each  session  began  with  hooking  up  the  eyetracker  and  calibrating  it  to  the 
individual  participant.  Next,  the  participant  was  shown  the  initial  screen.  All 
questions  were  asked  verbally  to  prevent  additional  eyetracking  to  the  question 
(Peebles  &  Cheng,  2001a,  2001b).  After  the  first  set  of  questions  was  finished, 
the  next  visualization  was  shown,  and  the  next  set  of  questions  was  asked.  When 
the  participant  finished  that  set,  a  blank  screen  was  shown  for  the  test  questions. 

3  Results 

How  do  skilled  users  extract  information  from  complex  visualizations?  Is  there 
a  difference  between  extractions  that  occur  when  the  information  is  explicitly 
available  on  the  visualization  and  when  the  information  is  not  explicitly  avail¬ 
able?  This  results  section  attempts  to  answer  these  questions. 


3.1  Overview 


All  forecasters  were  able  to  extract  the  information  from  the  graphs;  they  all 
knew  how  to  read  these  kinds  of  meteorological  visualizations. 

The  graph  comprehension  aspect  of  this  task  was  not  difficult  for  the  partici¬ 
pants;  experts  and  novices  were  very  accurate  when  reading  off  information  from 
the  graph.  There  were  no  consistent  qualitative  or  quantitative  differences  be¬ 
tween  experts  or  novices  during  graph  comprehension,  so  all  participants’  results 
were  combined. 


3.2  Extracting  information  that  is  explicitly  available 

We  first  examined  how  forecasters  extracted  information  that  is  explicitly  on  a 
complex  visualization.  For  several  of  the  questions  during  graph  comprehension, 
the  information  was  explicitly  available  on  the  graph.  Thus,  for  wind  speed  and 
wind  direction,  participants  simply  had  to  find  the  location  (e.g.,  Pittsburgh) 
and  read  off  the  wind  speed  and  wind  direction.  For  other  variables  (temperature 
and  relative  humidity)  the  forecasters  had  to  match  a  color  code  with  a  legend. 

Bertin’s  model  presented  earlier  suggested  that  after  people  encoded  a  graph¬ 
ical  element,  they  would  translate  the  elements  into  patterns  and  map  the  pat¬ 
terns  into  labels.  Bertin  (and  others)  implied  that  people  would  perform  this 
operation  serially.  Carpenter  &  Shah  (1998)  however,  found  that  participants 
interpreting  graphs  would  cycle  back  and  forth  between  the  graph  area  and 
the  legend,  suggesting  a  much  more  iterative  aspect  to  graph  comprehension 
than  previously  known.  We  explicitly  examined  if  experienced  (both  expert  and 
novice)  forecasters  would  cycle  back  and  forth  between  the  graph  area  and  the 
legend,  as  Carpenter  &  Shah  (1998)  found. 

During  the  graph  comprehension  stage,  we  asked  two  questions  whose  vari¬ 
ables  had  legends:  “What  is  the  temperature  at  Pittsburgh?”  and  “What  is  the 
relative  humidity  at  Honolulu?”  We  examined  the  number  of  times  that  partic¬ 
ipants  cycled  back  and  forth  between  the  graph  area  and  the  legend.  Figure  3 
shows  a  typical  subject’s  eye  movements  as  he  finds  the  answer  to  the  question 
“What  is  the  temperature  at  Pittsburgh,  Pennsylvania?” 

Consistent  with  Carpenter  &  Shah  (1998),  we  found  frequent  switching  for 
both  questions.  Table  1  shows  the  average  number  of  cycles  for  each  question. 


Question 

of  glances  between  graph  and  legend  (avg) 

What  is  the  temperature  at  Pittsburgh? 

6.0 

What  is  the  relative  humidity  at  Honolulu? 

4.5 

Table  1.  The  average  number  of  times  that  the  forecasters  looked  between  the  legend 
and  the  graph  area. 


Fig.  3.  Eyetracking  of  a  participant  after  asked  the  question,  “What  is  the  temperature 
at  Pittsburgh,  Pennsylvania?”  “Dots”  are  4  milliseconds  apart.  There  are  four  glances 
between  the  legend  and  the  graph  area  in  this  figure. 


3.3  Extracting  information  that  is  not  explicitly  available 

How  do  experienced  forecasters  extract  information  that  is  not  explicitly  avail¬ 
able  on  a  visualization?  There  are  some  propositional  theories  (e.g.,  Lohse,  1993; 
Peebles  &  Cheng,  2001a;  Pinker,  1990)  that  do  not  explain  the  extraction  process 
well  if  the  information  is  not  explicitly  available  on  the  graph.  At  a  minimum, 
these  propositional  theories  leave  this  process  unspecified.  Other  theories  (like 
Trafton  et  al.’s  (2001)  spatial  transformation  theory)  and  many  theories  on  di¬ 
agram  comprehension  suggest  that  people  use  both  propositional  and  spatial 
reasoning  (e.g.,  Tabachneck  et  al.’s  (1997)  CaMeRa  theory). 

In  order  to  examine  the  cognitive  process  of  how  experienced  graph  users  ex¬ 
tract  information  that  is  not  explicitly  available,  we  asked  experienced  forecast¬ 
ers  what  the  pressure  was  at  Pittsburgh.  Recall  that  pressure  is  not  represented 
explicitly  for  Pittsburgh;  forecasters  must  interpolate  between  the  two  pressure 
lines  to  determine  the  actual  pressure  of  Pittsburgh.  Eyetracking  results  show 
that  all  participants  went  through  several  steps  to  extract  this  information:3 

1.  Trace  the  upper  pressure  line; 

2.  Find  the  pressure  itself  (the  number); 

3  Note  that  this  was  the  fourth  question  asked  (the  third  about  Pittsburgh),  so  par¬ 
ticipants  already  knew  where  Pittsburgh  was. 


3.  Trace  the  lower  pressure  line; 

4.  Find  the  pressure  itself  (the  number); 

5.  Bridging  (glancing)  back  and  forth  between  the  two  pressure  lines. 

Table  2  shows  the  average  number  of  times  the  forecasters  performed  each 

tracing  task. 


Tracing  type 

^  of  Traces  (avg) 

Tracing  the  upper  pressure  line 

2.4 

Tracing  the  lower  pressure  line 

2.6 

Bridging  between  the  two  pressure  lines 

7.4 

Table  2.  The  average  number  of  times  that  the  forecasters  traced  each  pressure  line 
and  bridging  between  the  two  lines. 


Figure  4  shows  one  of  the  forecasters’  eye  movements  as  he  was  bridging 
between  the  two  lines.  We  interpret  these  eye  tracking  movements  to  suggest 
that  the  forecasters  were  using  the  diagram  to  help  them  calculate  the  distance 
between  the  two  lines.  We  believe  that  after  the  forecasters  located  the  pressure 
lines,  they  were  mentally  drawing  a  line  in  between  the  two  lines  and  dividing 
up  the  space  between  Pittsburgh  and  the  pressure  lines.  This  kind  of  spatial 
transformation  seems  to  be  more  effortful  than  simply  line  tracing,  as  shown 
by  the  increased  number  of  eye  tracking  “bridges”  between  the  two  pressure 
lines  as  compared  to  the  pressure  line  tracing,  \/2(2)  =  19.4, p  <  .001,  Bonferroni 
adjusted  y2s  significant  at  p  <  .05. 

3.4  Recalling  and  constructing  information 

After  the  graph  comprehension  part  of  the  task,  forecasters  were  asked  for  infor¬ 
mation  that  they  had  just  extracted  (Pittsburgh)  or  for  information  that  they 
had  seen  but  not  explicitly  extracted.  This  aspect  of  the  task  was  seen  as  quite 
difficult.  Several  participants  were  not  able  to  answer  some  of  the  recall  ques¬ 
tions,  especially  in  the  far  condition.  Experts  attempted  88%  of  the  questions, 
while  novices  attempted  only  56%  of  them,  \'2(1)  =  7.1  ,p  <  .01.  Figure  5  shows 
the  accuracy  of  those  participants  who  did  complete  the  questions  for  pressure 
(wind  speed,  wind  direction,  and  temperature  all  showed  a  similar  pattern). 
While  these  results  must  obviously  be  interpreted  with  care  because  of  the  low 
completion  rate,  it  is  evident  that  the  experts  were  far  more  accurate  than  the 
novices,  especially  at  the  non-recall  questions. 

A  very  small  number  of  forecasters  showed  an  interesting  pattern  while  an¬ 
swering  questions  about  temperature  (we  are  missing  some  eye  movement  data 
because  several  forecasters  closed  their  eyes  or  looked  off  the  screen).  When 
asked  about  the  temperature  for  Pittsburgh  (an  exact  question  that  was  being 
recalled),  most  forecasters  focused  on  a  specific  area  of  the  screen.  However, 
when  asked  about  temperature  of  other  areas  (near,  medium,  and  far),  67%  of 
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Fig.  4.  A  forecaster’s  eye  movements  as  he  was  bridging  between  the  two  pressure  lines. 
This  is  only  a  partial  display  of  his  eye  movements  while  answering  this  question  (i.e., 
the  tracing  the  two  pressure  lines  has  been  omitted. 


the  forecasters  “looked”  to  the  right  to  “examine”  the  legend,  even  though  there 
was  nothing  on  the  blank  screen  to  see.  We  interpreted  this  kind  of  glance  as 
evidence  for  visual  imagery.4 

Figure  6  shows  one  of  the  forecasters’  eye  movements  as  he  made  this  kind 
of  glance.  We  have  not  run  enough  participants  through  this  type  of  experiment 
or  even  seen  enough  evidence  of  this  kind  of  glance,  but  we  know  of  no  other  eye 
movement  study  that  has  demonstrated  any  kind  of  “examination”  of  a  mental 
image  by  eye  movements. 


4  General  Discussion 


In  this  paper,  we  explored  the  process  that  experienced  users  went  through  when 
they  extracted  information  from  a  complex  visualization  when  the  information 
was  explicitly  available  and  when  the  information  was  not  explicitly  available. 

4  It  is  interesting  that,  while  looking  at  the  legend  was  a  frequent  glance,  there  were 
other  glances  that  did  not  seem  to  happen.  For  example,  the  participants  only  rarely 
“looked”  at  a  location  that  was  geographically  correct,  though  there  was  some  change 
of  glance  location  when  questions  about  a  different  location  were  asked.  This  suggests 
that  the  entire  representation  is  not  encoded  exclusively  as  either  an  image  or  as  a 
proposition. 


Errors  made  on  the  pressure  questions 


Pressure 


Fig.  5.  Accuracy  of  the  forecasters  that  completed  the  question  for  the  pressure  ques¬ 
tion  across  all  locations. 


We  replicated  the  results  from  Carpenter  &  Shah  (1998)  and  showed  that 
even  experienced  users  scan  back  and  forth  between  a  legend  and  its  value. 

We  also  presented  eye  movement  data  that  showed  that  when  users  need 
to  interpolate  between  two  lines  (like  pressure  lines  or  isobars),  they  do  not 
use  a  propositional  representation;  rather,  they  use  some  spatial  or  imagerial 
process  to  trace  a  line  that  allows  them  to  extract  the  needed  information.  This 
extraction  literally  occurs  as  a  combination  between  the  internal  (visualization) 
and  external  representations  (spatial  transformations).  This  finding  also  provides 
further  support  for  the  spatial  transformation  framework  (Trafton  &  Trickett, 
2001;  Trafton  et  al.,  in  press). 

This  study  also  provided  some  very  preliminary  data  on  differences  between 
experts  and  novices.  We  found  that  both  expert  and  novice  forecasters  were  able 
to  accurately  extract  information  from  complex  meteorological  visualizations. 
We  also  found  that  experts  did  a  better  job  during  the  test  phase  for  non-recall 
items  than  novices.  This  finding  is  consistent  with  the  finding  of  Trafton  et  al. 
(2000)  who  suggested  that  experienced  forecasters  build  a  qualitative  mental 
model  (QMM)  and  reason  with  that  (rather  than  simply  using  climatological 
knowledge  or  memory).  This  finding  is  consistent  with  Trafton  et  al.’s  (2000) 


Fig.  6.  Eyetracking  of  a  participant  after  asked  the  question,  “What  is  the  temperature 
at  Philadelphia,  Pennsylvania?”  This  was  the  “Near”  question  during  the  recall  stage. 
The  screen  itself  was  blank;  it  showed  nothing  but  white,  so  there  was  literally  nothing 
for  the  forecaster  to  look  at. 


view  that  expert  forecasters  are  able  to  generate  complex  quantitative  relation¬ 
ships  by  extracting  primarily  qualitative  information  from  a  complex  weather 
visualization. 

We  also  suggested  that  some  forecasters  use  visual  imagery  to  generate  in¬ 
formation  they  can  not  recall.  Other  researchers  have  suggested  that  visual  or 
spatial  imagery  is  used  when  reasoning  with  complex  diagrams  (e.g.,  Hegarty 
&  Sims,  1994;  Tabachneck-Schijf  et  al.,  1997),  but  most  theories  of  graph  com¬ 
prehension  do  not.  One  of  our  main  goals  in  this  paper  was  to  provide  some 
evidence  for  combining  some  of  the  visual  and  spatial  research  from  diagram¬ 
matic  reasoning  into  the  graphical  reasoning  research. 
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